3 research outputs found
Anchorage: Visual Analysis of Satisfaction in Customer Service Videos via Anchor Events
Delivering customer services through video communications has brought new
opportunities to analyze customer satisfaction for quality management. However,
due to the lack of reliable self-reported responses, service providers are
troubled by the inadequate estimation of customer services and the tedious
investigation into multimodal video recordings. We introduce Anchorage, a
visual analytics system to evaluate customer satisfaction by summarizing
multimodal behavioral features in customer service videos and revealing
abnormal operations in the service process. We leverage the semantically
meaningful operations to introduce structured event understanding into videos
which help service providers quickly navigate to events of their interest.
Anchorage supports a comprehensive evaluation of customer satisfaction from the
service and operation levels and efficient analysis of customer behavioral
dynamics via multifaceted visualization views. We extensively evaluate
Anchorage through a case study and a carefully-designed user study. The results
demonstrate its effectiveness and usability in assessing customer satisfaction
using customer service videos. We found that introducing event contexts in
assessing customer satisfaction can enhance its performance without
compromising annotation precision. Our approach can be adapted in situations
where unlabelled and unstructured videos are collected along with sequential
records.Comment: 13 pages. A preprint version of a publication at IEEE Transactions on
Visualization and Computer Graphics (TVCG), 202
M2Lens: Visualizing and explaining multimodal models for sentiment analysis
Multimodal sentiment analysis aims to recognize people's attitudes from
multiple communication channels such as verbal content (i.e., text), voice, and
facial expressions. It has become a vibrant and important research topic in
natural language processing. Much research focuses on modeling the complex
intra- and inter-modal interactions between different communication channels.
However, current multimodal models with strong performance are often
deep-learning-based techniques and work like black boxes. It is not clear how
models utilize multimodal information for sentiment predictions. Despite recent
advances in techniques for enhancing the explainability of machine learning
models, they often target unimodal scenarios (e.g., images, sentences), and
little research has been done on explaining multimodal models. In this paper,
we present an interactive visual analytics system, M2Lens, to visualize and
explain multimodal models for sentiment analysis. M2Lens provides explanations
on intra- and inter-modal interactions at the global, subset, and local levels.
Specifically, it summarizes the influence of three typical interaction types
(i.e., dominance, complement, and conflict) on the model predictions. Moreover,
M2Lens identifies frequent and influential multimodal features and supports the
multi-faceted exploration of model behaviors from language, acoustic, and
visual modalities. Through two case studies and expert interviews, we
demonstrate our system can help users gain deep insights into the multimodal
models for sentiment analysis.Comment: 11 pages, 7 figures. This paper is accepted by IEEE VIS, 2021. To
appear in IEEE Transactions on Visualization and Computer Graphics (TVCG